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ABSTRACT 


Techniques  for  the  detection  of  a  weak  signal  in  non-Gaussian.  ill-defined 
noise  are  considered.  Statistical  characterizations  used  are  moments,  tail 
measures  related  to  quantiles,  and  a  measure  related  to  the  score  function.  For 
multivariate  densities,  the  characterization  is  by  means  of  a  nonlinear  transfor¬ 
mation.  Initial  results  seem  to  indicate  that  assuming  a  particular  family  of  pro¬ 
bability  densities  does  not  necessarily  result  in  a  significant  degradation  in  per¬ 
formance  when  the  observations  actually  come  from  a  density  outside  the 
assumed  family.  More  important  to  performance  are  accurate  estimates  of  the 
moments,  tail  measures,  or  other  parameters  which  are  used  to  specify  the 
detector. 

INTRODUCTION 

In  most  engineering  studies,  there  is  usually  a  tradeoff  between  model  com¬ 
plexity  and  analytical  tractability:  the  more  complicated  (and  realistic)  the 
model  is  assumed  to  be,  the  more  difficult  the  subsequent  analysis  becomes. 
This  balance  is  especially  delicate  in  the  area  of  non-Gaussian  signal  processing. 


If  departures  from  Gaussian  statistics  are  investigated,  it  is  usually  assumed 
that  the  sequence  of  observations  is  independent  and,  often,  identically  distri¬ 
buted.  With  these  basic  assumptions,  analytical  results  are  often  available.  When 
non-Gaussian  dependencies  are  taken  into  account,  results  are  often  only 
obtainable  by  means  of  Monte  Carlo  simulation  since  multivariate  distributions 
are  usually  not  available  in  analytical  form. 

It  is  within  this  framework  that  we  summarize  some  of  our  preliminary 
results  for  detection  in  a  non-Gaussian  environment.  We  first  consider  the  detec¬ 
tion  of  a  signed  in  nearly  Gaussian  skewed  noise.  Surprisingly,  a  small  degree  of 
skewness  can  lead  to  a  significant  degradation  in  performance  of  the  linear 
detector  as  measured  by  false-alarm  rate.  A  detector  which  exhibits  the  desired 
robustness  is  introduced.  Some  of  the  potential  difficulties  in  using  adaptive 
procedures  are  illustrated  in  the  context  of  under-ice  ambient  noise  data. 

A  general  adaptive  procedure  is  then  outlined  which  uses  the  skewness- 
kurtosis  plane  to  measure  departures  from  Gaussian  statistics.  Overlays,  which 
specify  the  probability  density  family  of  the  observations,  are  introduced  and 
used  to  determine  the  form  of  the  nonlinear  processor. 

Non-Gaussian  statistics  are  then  characterized  by  the  derivative  of  the  loga¬ 
rithm  of  the  probability  density.  This  expression,  f'(x)/f(x),  is  estimated  using 
the  observations  and  is  then  used  to  form  an  optimum  detector.  It  is  shown  by 
example  that,  under  reasonable  conditions,  assuming  a  particular  family  of  pro- 
babilty  densities  does  not  significantly  degrade  detector  performance  when  the 
observations  actually  come  from  a  density  outside  the  family.  This  general  con¬ 
clusion  is  also  arrived  at  using  another  approach,  in  which  quantiles  are  used  to 
measure  the  tail  behavior  of  heavy-tailed  probability  densities. 

In  the  last  section,  a  class  of  multivariate  non-Gaussian  probability  densi¬ 
ties  is  defined.  For  this  class,  the  locally  optimum  detector  is  seen  to  separate 
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into  a  zero-memory  nonlinear  part  and  a  part  with  memory.  The  results  of  a 
simulation  to  evaluate  a  number  of  detectors  are  presented.  It  is  observed  that 
simplified  versions  of  the  optimal  processor  also  lead  to  improved  performance 
when  compared  to  conventional  detectors. 

DETECTION  IN  SKEWED  NOISE 

In  robust  and  general  nonparametric  studies  (Refs.  1,2).  a  symmetry 
assumption  on  the  underlying  noise  density  is  usually  made.  (Another  frequent 
nonparametric  assumption  is  that  the  noise  density  is  unimodal  or  that  the 
median  is  1/2.)  In  this  section,  we  report  on  an  investigation  which  assumes 
nonzero  skewness  and  which  studies  the  sensitivity  of  the  linear  and  sign  detec¬ 
tors  to  this  lack  of  symmetry.  Surprisingly,  a  small  amount  of  skewness  can 
lead  to  marked  deterioration  in  system  performance  for  the  linear  detector  as 
measured  by  false-alarm  rate.  A  modified  detector  is  also  introduced  which 
exhibits  a  desired  robustness  to  skewness  in  the  observations. 

Consider  the  detection  problem 

H0-  Xi  =  Nt  ,  i=l,2,...* 

Hi.  + 

where  the  constant  signal  9>0.  The  noise  is  independent, identically  distributed 
with  first  three  moments 

£W= 0.  m8)=J*.  E(Xia)=^>  0 
The  skewness  is  defined  as 

t  =  Ha/o* 

and  is  assumed  to  be  small. 

If  skewness  were  zero  and  the  noise  Gaussian,  the  optimum  receiver  is  the 
sample  mean  or  linear  detector 


r»<*)  =  rt  Xi 

*  <■! 
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With  a  small  amount  of  skewness  and  nearly  Gaussian  noise,  the  test  statistic  T 
can  be  represented  by  the  first  few  terms  of  the  Cornish-Fisher  expansion 
(Refs.3,4).  Under  hypothesis  Ho 

Tz{x)=^-Z  +  ~^-(Z8-l)+0(*"a/2) 

where  Z  is  the  standard  normal  variable.  (Under  hypothesis  Hx  the  above 
expression  includes  a  shift  by  the  signal.) 

Utilizing  the  first  two  terms  of  this  expansion,  one  can  generate  the  proba¬ 
bility  density  function  for  To,  along  with  analytic  expressions  for  the  false-alarm 
rate  and  detectability.  Details  can  be  found  in  [3],  [4]. 

Figure  1  compares  the  Gaussian  density  to  the  density  of  Te  with  skewness 
(-0.5.  Figure  2  gives  the  normalized  false-alarm  rate  as  a  function  of  skewness. 
The  number  of  observations  for  the  test  is  100.  The  constant  Oq  is  the  false- 
alarm  rate  under  the  strict  Gaussian  assumption  (f=0)  and  provides  the  thres¬ 
hold  setting  for  Ta.  A  slight  departure  from  symmetry  is  hardly  discernible 
(Fig.l).  Yet.  for  (=0.5.  there  is  an  increase  in  false-alarm  rate  of  over  80%  for  a 
nominal  a^lO-8.  With  smaller  a0.  the  degradation  is  even  more  severe  (Ref.  4). 

Clearly,  the  linear  detector  can  be  modified  so  as  to  account  for  skewness. 
One  approach  is  to  use  a  nonparametric  test  such  as  the  sign  detector,  which 
keeps  the  false  alarm  rate  relatively  constant.  (See  Ref.  3  for  details.)  A  second 
approach  is  to  directly  modify  the  linear  detector.  Here,  a  natural  way  to 
proceed  would  be  to  "subtract  off"  that  part  of  the  observation  due  to  skewness. 
The  following  test  statistic  is  the  simplest  version  of  this  approach: 

Ta(x)  =  Tx{z)  -  £-(7\(*))8 

where  Tx  is  the  sample  mean  detector  given  above.  Analytic  expressions  can 
again  be  developed  for  false  alarm-rate  and  power  (Ref. 4).  Here,  we  choose  to 
present  the  results  of  a  Monte  Carlo  simulation  which  also  serves  to  verify  the 
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accuracy  of  the  analytic  approximations.  The  results  for  false- alarm  rates  in 
Fig.  3  indicates  that  the  modified  sample  mean  detector  has  the  desired  robust¬ 
ness. 

implicit  in  the  improved  performance  is  the  ability  to  measure  the  skew¬ 
ness  accurately.  This  is  clearly  illustrated  with  a  simple  experiment  performed 
with  under-ice  ambient  noise.  Fram  II  data  (Ref.5  and  also  discussed  elsewhere 
in  these  Proceedings)  was  used  to  compute  empirically  the  false-alarm  rate  for 
the  linear  detector  and  the  modified  version  discussed  above.  Required  esti¬ 
mates  of  the  variance  and  skewness  were  obtained  by  straightforward  sample 
moment  methods  over  (assumed  stationary)  blocks  of  data.  Figure  4  summar¬ 
izes  the  results  of  our  first  experiment.  Performance  of  both  detectors  is  essen¬ 
tially  the  same,  in  sharp  contrast  to  the  results  obtained  from  computer¬ 
generated  data  discussed  in  the  above  paragraph-  The  primary  reason  for  this 
is,  we  suspect,  related  to  the  nature  of  the  nonstationarity  of  the  data  (See  Ref. 
5,  page  6  and  Fig.  5  below)  and  the  need  to  use  more  accurate  estimates  of 
skewness  with  better  tracking  properties.  This  aspect  of  data-adaptive  estima¬ 
tion  and  detection  is  an  area  of  current  research  activity. 

To  conclude  this  section,  we  will  outline  one  possible  adaptive  system  which 
uses  sample  moments.  Computed  skewness  and  kurtosis  are  shown  versus  time 
for  a  representative  Fram  II  data  set  in  Figs.  5,6.  (See  Ref.  7  for  further  details.) 
Kurtosis  is  defined  as  the  normalized  fourth  central  moment: 

P z-  E{X-m)*/  a* 

Figure  7  presents  the  same  skewness  (0i)  versus  kurtosis  (fa),  with  time  an 
implicit  parameter.  Observe  the  cluster  of  points  around  /9t=O,08=3,  which  are 
the  values  for  a  Gaussian  density.  The  overlaid  lines  are  different  regions  of  the 
Johnson  family  of  densities.  (The  Johnson  family  can  be  defined  as  a  nonlinear 
transformation  of  Gaussian  variates.  The  symbol  Su,  for  example,  represents 
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the  sinh  transformation,  while  St  is  a  logarithmic  transformation,  leading  to  a 
lognormal  random  variable.  (See  Ref.  6,  section  2.2  for  further  details.) 

Each  point  in  the  skewness-kurtosis  plane  defines  a  unique  member  of  the 
Johnson  family.  This  one-to-one  relationship  can  be  utilized  in  a  data-adaptive 
detector  in  the  following  manner.  A  region  around  the  Gaussian  point 
(0,=O./?2=3)  can  be  defined.  For  points  lying  in  this  region,  it  is  assumed  that 
the  observations  are  governed  by  Gaussian  statistics  and  the  optimum  processor 
is  the  linear  detector.  When  the  computed  moments  fall  outside  the  region,  one 
declares  that  the  observations  are  non-Gaussian  and  another  detector  is 

I 

switched  in  to  process  the  data.  The  point  in  the  Pi-fa  plane  would  determine 

t 

the  appropriate  density  which  then  specifies  the  likelihood  processor  (locally 
optimum  detector  in  the  weak  signal  case.)  In  practice,  the  Pi~Pz  plane  would 
be  quantized  into,  sajr,  rectangular  regions.  Then,  either  due  to  nonstationary 
statistics  or  because  of  sampling  variances  of  the  moment  or  other  estimators, 
the  point  wanders  around  in  the  region.  When  it  leaves  one  region,  another  likel¬ 
ihood  (nonlinear)  processor  can  be  be  switched  in  Clearly,  there  are  two  key 
steps.  The  first  is  to  obtain  good  tracking  estimates.  The  second  is  to  specify 
which  family  of  probability  densities  to  overlay  on  the  Pi~Pg  plane,  e.g.,  the 
Johnson.  Pearson,  or  mixture  model.  This  point  will  be  discussed  in  more  detail 
in  the  next  section 

DENSITY  FAMILIES  AND  ADAPTIVE  DETECTION 

Rather  than  focus  on  a  particular  moment  measure  such  as  skewness  or 
kurtosis  to  charcterize  non-Gaussian  statistics,  one  can  attempt  to  generalize, 
but  still  parameterize,  the  problem  in  the  following  manner. 

The  score  function  of  the  density  f(x)  is  related  to  f’(x)/f(x)  and  plays  an 
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important  role  in  estimation  theory  [8].  In  addition,  -f’(x)/f(x)  determines  the 
form  of  the  processor  for  the  locally  most  powerful  test  or  the  detection  of  a 
weak  signal  in  noise.  (This  expression  is  also  related  to  the  test  which  maxim¬ 
izes  efficacy  in  the  small  signal  case.) 

These  observations  lead  us  to  focus  on  using  f'/f  as  a  characterization  of 
non-Gaussian  statistics.  Observe  that  for  the  Gaussian  density  f'/f  is  linear,  so 
departures  from  Gaussian  statistics  can  be  conveniently  measured  by  depar¬ 
tures  from  linearity.  A  straightforward  way  to  proceed  is  to  specify  f'/f  as  a 
ratio  of  two  polynomials,  Specializing  to  first-order  numerator  and  second-order 
denominator  polynomials  gives  the  classical  Pearson  family  of  probability  densi¬ 
ties: 

Llx±=  _ _ 

/(*)  bg+bjZ+bsZ^ 

which  includes  the  Gaussian,  gamma,  and  beta  among  its  members. 

The  procedure  would  now  be  to  use  the  observations  to  estimate  the  above 
coefficients.  This  determines  the  particular  density  of  the  Pearson  family  and 
the  corresponding  detector.  The  two  main  approaches  for  estimating  the 
coefficients  are  maximum  likelihood  estimates  of  the  paramters  directly,  and 
indirect  estimates  using  sample  moments. 

Of  immediate  interest  is  the  question:  what  happens  when  there  is  a 
mismatch.  i.e„  when  we  use  data  generated  by  a  non-Pearson  density  to  fit  a 
Pearson-type  detector  as  described  above?  We  proceed,  using  the  non-Pearson 
mixture  model  ([7]): 

/(*)  =  (1-e) 

The  ratio  of  the  component  variances  is  defined  to  be 

y  -  (a a)2/  2 

This  mixture  density  has  been  used  to  model  heavy-tailed  Impulsive  noise.  (See 
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Ref.  9  and  the  references  cited  therein.) 

For  the  results  that  follow,  the  first  four  moments  of  toe  mixture  model  are 
used  to  obtain  analytically  the  parameters  a. 6 o.&i,i>2.  This  then,  determines  the 
processor,  in  this  case  the  locally  optimum  detector.  Figure  8  shows  the  curve 
g(x)=-f'(x)/f(x)  which  is  the  zero-memory  nonlinear  part  of  the  locally  optimum 
detector.  This  is  for  a  symmetric  density  (o=bi=0),  with  kurtosis  shown  as  a 
parameter.  Figures  9a-9d  present  a  comparison  of  ARE.  as  a  function  of  the 
mixing  constant  t,  with  y  as  a  parameter.  (The  ARE.  asymptotic  relative 
efficiency,  is  the  ratio  of  efficacies.  Efficacies,  in  turn,  are  incremental  signal- 
to-noise  ratio  measures  in  the  small  signal  case.)  ARE(lo.ld)  is  the  ratio  of  the 
locally  optimum  to  the  linear  detector,  while  ARE(g,ld)  compares  the  Pearson 
fit  as  discussed  above  to  the  linear  detector.  It  is  seen  that  for  small  e  and  y. 
the  Pearson-fit  detector  compares  quite  favorably  to  the  locally  optimum,  even 
though  g(x)  was  determined  from  a  non-Pearson  density.  (Additional  details 
and  further  examples  can  be  found  in  Ref.  7.) 

Based  on  these  preliminary  computions,  it  would  appear  that,  for  nearly 
Gaussian  noise  (a  mixing  parameter  of  t  <  0. 1).  the  assumption  of  operating 
within  a  Pearson  family  does  not  significantly  degrade  detector  performance 
when  the  true  density  turns  out  to  be  a  Gaussian  mixture  with  a  Laplace  contam¬ 
inant. 

One  of  the  difficulties  with  moment  estimators  is  the  potentially  large  sam¬ 
pling  variance  for  the  higher-order  moments.  Oftentimes,  it  is  more  appropriate 
to  use  quantile  measures  to  characterize  the  underlying  statistics.  We  now  out¬ 
line  a  study  using  these  measures.  (See  Refs.  10  and  11  for  details.) 

Let  F(x)  be  the  cumulative  distribution  function  for  the  noise.  Let  pi 
represent  a  quantile  on  the  tail  of  the  density,  e  g.,  pi=.999,  and  let  Ps<Pi  be  a 
lower  quantile.  Then,  two  other  measures  which  can  be  used  to  characterize  the 
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density  (and  its  tails)  are: 

r  =  /’-1(p,)/JP-t(p2) 

and 

p=  In  F-'iPi) 
dp  1 

Figure  10  illustrates  the  one-to-one  relationship  between  these  measures  (and 
kurtosis)  and  the  Johnson  Su  family  as  one  of  the  two  parameters  6  of  this  den¬ 
sity  varies.  The  other  parameter  is  determined  from  the  variance  (measured  or 
given)  and  6.  (See  [10]  or  [11],  Chapt.  4.) 

Using  these  measures,  one  can  parallel  the  adaptive  procedure  outlined 
above.  That  is,  the  data  is  used  to  estimate  <5.  This,  in  combination  with  an  esti¬ 
mate  of  o,  uniquely  determines  the  Johnson  probability  density  and,  hence,  the 
nonlinear  detector.  This  has  been  simulated  when  the  noise  does  not  actually 
come  from  a  Johnson  density  but,  rather,  from  other  heavy-tailed  densities 
(Gaussian-Gaussian  mixtures,  Lapacian  noise).  The  results  of  a  preliminary  com¬ 
puter  simulation  are  encouraging:  the  detector  based  on  Johnson  statistics  per¬ 
forms  close  to  the  optimum  detector  (which  is  based  on  exact  knowledge  of  the 
noise  statistics)  and  always  better  than  the  linear  detector  ([11]).  Again  we  see 
that  detector  performance  does  not  appear  to  be  critically  dependent  on  speci¬ 
fying  the  correct  family  of  densities.  More  important,  apparently,  are  accurate 
estimates  of  the  parameters  which  characterize  the  family  and  the  correspond¬ 
ing  detector 

DETECTION  IN  MULTIVARIATE  NOISE 

The  previous  discussion  assumed  the  observations  were  independent;  in 
order  to  account  for  the  dependencies,  multivariate  probability  densities  need 
to  be  considered.  In  this  section,  we  study  nonlinear  transformations  of  a  known 
multivariate  density.  This  type  of  noise,  called  transformation  noise  in  Refs.  7 
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and  12,  is  a  multivariate  generalization  of  the  Johnson  family  discussed  earlier. 
Any  output  marginal  can  be  obtained  by  presenting  a  zero-memory  nonlinear 
transformation.  It  is  difficult,  however,  to  obtain  analytically  the  output  depen¬ 
dencies.  Figure  11  is  a  schematic  of  the  transformation  noise  generation.  The 
output  multivariate  density  is  denoted  by  /  (a)  and  the  input  density  by  59(14). 
With  u.  multivariate  Gaussian,  there  is  a  further  decomposition,  since  one  can 
easily  define  the  linear  transformation  11. =Lz.,  where  2.  is  a  vector  of  independent 
Gaussian  variates,  andX  defines  the  covariance  structure. 

For  the  case  of  a  weak  signal  in  noise,  the  appropriate  receiver  is  the  locally 
optimum  detector.  It  is  shown  in  Fig.  12,  where  it  is  assumed  that  the  nonlinear¬ 
ity  g  is  twice  differentiable.  The  symbol  0  denotes  element-by-element  vector 
multiplication  and  0  is  the  vector  dot  product.  Observe  that  the  detector  con¬ 
sists  of  a  zero-memory  nonlinear  part  (g,g\  etc...)  and  the  locally  optimal  non¬ 
linearity  with  memory,  V59/50,  for  a  signal  in  noise  with  a  density  59.  With  ji  mul¬ 
tivariate  Gaussian,  V59/  <f>  reduces  to  the  usual  linear  matched  filter,  /?"lx. 

Figure  13  summarizes  the  results  of  a  Monte  Carlo  simulation  to  evaluate 
ARE.  (See  [7], [12]  for  details.)  The  output  marginal  was  specified  as  Laplacian: 

/  i(n)  =  ~exp(—a  |n  |) 

which  was  generated  by  a  suitable  transformation  of  multivariate  Gaussian 
noise.  The  Gaussian  vector  was  assumed  to  be  m-dependent,  with  the  correla¬ 
tion  function  taken  as  triangular: 

Pi  -  1-  \i/m  |,  |i  |  s  m 
Pi  -  0,  \i  \  >m 

The  nonlinearities  required  in  the  detector  are  determined  from  /j(n),  and  the 
signal  was  taken  as  a  constant. 

Four  detectors  were  simulated.  TTie  first  was  the  locally  optimum  detector, 
and  the  second  assumed  independent  noise.  The  third  was  a  simplification  of 
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the  first;  the  indicated  vector  multiplication  was  removed.  The  fourth  detector 
was  the  linear  matched  filter.  As  indicated  above,  values  for  the  ARE  were  com¬ 
puted  via  simulation;  consequently,  it  is  difficult  to  make  general  statements. 
Nevertheless,  some  observations  seem  appropriate. 

It  is  clear  that  for  small  correlation  time,  m  s  3  or  4,  an  independence 
assumption  does  not  significantly  degrade  detector  performance.  Second,  with 
some  sort  of  nonlinear  processing,  i.e.,  taking  into  account  non-Gaussian  statis¬ 
tics,  a  reasonable  improvement  can  usually  be  obtained:  for  any  of  the  nonlinear 
detectors  compared  to  the  linear  one,  the  ARE  for  this  simulation  does  not  fall 
below  2.  Note  that  detectors  2  and  3  give  similar  ARE  values.  This  is  interesting 
since  one  detector  assumes  independent  noise,  while  the  other  simplfies  the 
optimal  detector,  but  keeps  the  dependency  assumption.  Clearly,  one  would  like 
to  learn  what  are  the  essential  common  features  of  the  various  detectors  that 
lead  to  improved  performance.  Then  only  these  features  need  be  incorporated 
into  a  practical  receiver. 

SUMMARY 

Techniques  for  detection  of  weak  signals  in  non-Gaussian  noise  have  been 
considered.  The  importance  of  both  learning  and  robust  procedures  was  illus¬ 
trated  by  means  of  an  example  where  a  modest  deviation  from  a  Gaussian  noise 
assumption  (in  skewness)  led  to  a  substantial  increase  in  the  false  alarm  rate  for 
a  linear  detector.  In  contrast,  a  modified  (non-linear)  detector  was  robust  and 
maintained  a  relatively  constant  false  alarm  rate  for  a  wide  range  of  skewness. 
These  analytical  results  were  verified  by  computer  simulation. 

Under-ice  ambient  noise  was  used  in  an  experiment  to  illustrate  some  of 
the  practical  difficulties  with  adaptive  procedures;  in  this  case,  because  of  the 
non-stationarity  of  the  noise,  there  was  a  need  to  incorporate  into  the  adaptive 
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detector  parameter  estimates  with  enhanced  tracking  properties. 

The  skewness-kurtosis  plane  was  presented  as  a  convenient  graphical 
display  to  place  in  evidence  the  time-varying  nature  of  the  non-Gaussian  statis¬ 
tics.  The  notion  of  overlays  was  also  introduced  to  define  density  families  and 
the  corresponding  likelihood  processors.  Densities  were  also  characterized  in 
terms  of  a  function  related  to  the  score  function  and  tail  measures  using  quan¬ 
tiles.  Our  preliminary  results  seem  to  indicate  that  detector  performance  does 
not  appear  to  be  critically  dependent  on  specifying  the  correct  family  of  densi¬ 
ties.  More  important  to  performance  are  accurate  estimates  of  the  moments, 
tail  measures,  or  other  parameters  which  are  used  to  specify  the  detector. 

A  particular  class  of  multivariate  non-Gaussian  densities  was  defined  and 
the  canonical  form  of  the  locally  optimum  detector  derived.  The  results  of  a 
simulation  to  evaluate  ARE  for  the  optimum  detector  and  simplified  versions 
were  presented.  111086  results  indicate  that  some  sort  of  nonlinear  processing 
which  takes  into  account  deviations  from  Gaussian  statistics  leads  to  a  large 
portion  of  the  improved  performance. 
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